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ABSTRACT 



This paper describes a Personal Intelligent Mentor (PIM) 
that facilitates metacognitive development in the domain of solving logic 
word puzzles. Metacognition is an important aspect for critical thinking 
skills. High school students must develop logical and critical thinking 
abilities as a prerequisite for higher-level math and computer programming 
classes. There is substantial interest at present in the development and use 
of software agents. They are being deployed in large and increasing numbers 
to assist users in finding and managing information. This research focuses on 
how intelligent agents can help students to improve their problem solving 
skills. The paper first motivates the utility and need for agents for 
education. Using this motivation as a guide, requirements are derived for the 
formation of a PIM as a cognitive tool. The paper then describes possible 
tasks for the Personal Intelligent Mentor and discusses how the PIM will 
interact with other agents, its environment, and users to improve its 
capabilities. (Contains 19 references.) (Author/AEF) 
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Abstract: Metacognition is an important aspect for critical thinking skills. This paper describes a 
Personal Intelligent Mentor (PIM) that facilitates metacognitive development in the domain of solving 
logic word puzzles. There is substantial interest at present in the development and use of software 
agents. They are being deployed in large and increasing numbers to assist users in finding and managing 
information. Our research focuses on how intelligent agents can help students to improve their problem 
solving skills. The paper first motivates the utility and the need for agents for education. Using this 
motivation as a guide, we derive requirements for the formation of a PIM as a cognitive tool. We then 
describe possible tasks for the Personal Intelligent Mentor and finally we discuss how the PIM will 
interact with other agents, its environment and user to improve its capabilities. 
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1 Introduction 

The focus of this research is to promote metacognition. Schoenfeld (1987) describes three 
categories of metacognition: 1) knowledge of one's thought processes; 2) control or self- 
regulation; 3) beliefs or intuitions. For the purposes of this research, we will focus on the first 
and second categories: that metacognition incorporates knowledge and monitoring of one's 
thought processes. Information processing models of cognition (e.g., Pressley & McCormick, 
1995) suggest the primary importance of metacognitive skills, particularly as metacognitive 
ability is a feature of expert problem solvers (Glaser & Chi, 1988). Whereas novices tend to 
focus on surface features of a problem, experts tend to better organize and represent the 
information and use more metacognitive skills. Although a metacognitive state may hinder 
intuitive ability (Baylor, in press), it is beneficial for other reasoning processes. 

One particularly salient reasoning process that depends on metacognitive skills is logical 
reasoning. High school students must develop logical and critical thinking abilities as a 
prerequisite for higher-level math (e.g. advanced algebra, trigonometry, calculus), science 
(e.g. chemistry, physics, applied biology), and computer programming classes. Part of the 
problem with a traditional approach to teaching logic is that logic, by its very nature, is 
abstract. When metacognitive reasoning of logical reasoning is considered, it is very abstract! 
We chose to research high school students' metacognitive reasoning with logic word puzzles 
(see example in Appendix A) because they are concrete, well-defined, and are more 
motivational for the students than simply answering questions about logic. 

The primary goal of this research is to give high school students practice in thinking logically 
and developing problem-solving strategies through the process of solving logic word puzzles 
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using an intelligent agent (specifically, a Personal Intelligent Mentor (PIM)) as a cognitive 
tool. 



2 The role of artificial intelligence 

In general, an intelligent agent can be defined as a software program that performs some tasks 
in an environment with autonomous processing capabilities, so "any property of an agent must 
therefore be defined in terms of the task and the environment in which the task is to be 
performed" (Goodwin, 1993). From an educational vantage point, a better description might 
be that intelligent agents are computer programs that simulate a human relationship by doing 
something that another person could otherwise do for you (Seiker, 1994, p.92). 

The goal of our research is that as a result of solving the logic puzzles interactively with the 
intelligent agent (i.e., Personal Intelligent Mentor), students will improve their semantic 
understanding of logic and their logical problem-solving skills. Throughout the problem- 
solving process, the PIM will facilitate metacognition and promote mindfulness while the 
student is problem-solving. 

In considering the use of intelligent systems for education, there are generally two camps 
(Lajoie & Derry, 1993): 1) those who promote using the system as a cognitive tool to 
stimulate the student to monitor and diagnose performance; and, 2) those who promote using 
the model building approach to use the system as an intelligent tutor. In our approach, the 
agent will function more as an intelligent cognitive tool than as an intelligent tutor or assistant. 



3 The Personal Intelligent Mentor (PIM) as a cognitive tool 

Cognitive tools (e.g. Lajoie & Derry, 1993) are mental and computational devices that 
support, guide, and extend the thinking processes of their students (Derry, 1990). The agent in 
this project will serve a similar function. From a learning theory perspective, the concept of 
distributed cognitions (Salomon, 1993b) readily applies to intelligent agents since they could 
be used to serve as extensions of a person's intellectual capacity. Similarly, Vygotsky (1962) 
defines a person's zone of proximal development to be the limit for his/her ability to imitate 
processes demonstrated by others. By extending the cognitive capabilities of the person, 
intelligent agents could serve to thus augment a person's zone of proximal development. 

For an intelligent agent to facilitate the learning process, it is necessary for the student to 
actively use the agent as a cognitive tool rather than passively letting the agent retrieve 
information. In this way the intelligent agent provides an environment where the learner must 
think harder and more deeply about the content, using the agent as a natural cognitive 
extension. This research allows for intelligent agents to support an individual's metacognitive 
processes with the agents serving as a technological "reciprocal teacher" (e.g. Palinscar & 
Brown, 1984), prompting the individual to engage in analysis of his/her own cognitive 
processes. This tool serves to encourage the individual to assess what cognitive strategies are 
being used, similar to Salomon's pedagogic computer program, the Writing Partner (Salomon, 
1993a), which asks the learner intelligent questions through the writing process. 

Aside from serving as a cognitive tool, an additional feature is that the intelligent agents can 




3 



be developed to adapt behavior based on the student's history of performances. In this way, the 
agents could analyze the student's approach to a task, build a database of past activities, and 
provide suggestions of better strategies. In effect, the agents learn the way that the student 
learns. In this manner, the agents could infer expected behavior and advise the student 
accordingly. A critical issue in terms of educational value is in moderating between the agent 
taking over thinking for the student with the agent training the student to think more 
effectively. Salomon (1993a) refers to this as the difference between the effects "of' and 
"with" technology, with effects "with" technology being more desirable. We address this by 
providing a monitor serving the primary purpose of determining the correct amount of 
feedback. 

Before we define the properties of the Personal Intelligent Mentor (PIM), we will describe its 
capabilities and environment, since the properties of an agent depend on its environment and 
task. A task is a description of what the agent is supposed to achieve in the environment. The 
following section describes possible tasks for the Personal Intelligent Mentor based on 
educational principles. 



4 Possible tasks for Personal Intelligent Mentor (PIM) 

4.1 Serve as a pedagogical expert 

The PIM will monitor and evaluate the timing and implementation of teaching interventions 
(e.g. help, feedback). McArthur, Lewis and Bishay (1993) state that the pedagogical 
component of intelligent systems receives relatively little mention with current systems 
demonstrating little pedagogical expertise. As they suggest, most intelligent tutoring systems 
are constrained to a single method of teaching and learning, while truly expert human tutors 
can adopt different methods. Drawing from human research on expert tutoring, the PIM will 
be able to mentor flexibly, as a human mentor would. 

In terms of developing a relationship with the student, the PIM will obviously need to be 
trustworthy, honest, and cooperative while providing feedback. An important part of this 
feature is that the PIM will resemble a human tutor in terms of motivational qualities. As 
Lepper & Chabay (1987) propose, motivational components are as important as cognitive 
components for an intelligent tutor. Taking it one step further, they propose that bringing 
empathy to computer tutors is conducive to learning. The PIM is developed with these 
considerations in mind. 

The relationship of the PIM to the student can be described as a cognitive apprenticeship, 
where the student improves his/her performance while working with the more expert 
performer: the Personal Intelligent Mentor. In support of this feature, Collins & Brown (1987) 
suggest that students may learn best in environments that include agents which can intensively 
model and coach formative skills. Under our approach, as the student gains expertise, the 
agent would fade and allow for more student initiative. 

4.2 Evaluate student's progress in terms of logical reasoning 

While there is flexibility for accommodating different problem-solving approaches and 
preferences, the PIM will apply basic rules of logic to evaluate the student's logical reasoning. 
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For example, this task of the PIM would be to identify typical errors in logical reasoning, such 
as "modus tollens," where the student wrongly infers that "if A then B "and "B is false" then 
"A is false". 

The PIM will also provide a detailed error analysis of student progress. 

4.3 Assess student progress 

The PIM will determine student progress in the problem, in terms of correctly-eliminated 
logically contradicted cases, and a clue by clue analysis. 

4.4 Promote reflection and mindfulness 

COACH is an intelligent agent system by Seiker (1994) that records user experience to create 
personalized user help for LISP with an adaptive interactive help system. As Seiker describes, 
(p. 92) "Just as a football coach will stand on the sidelines and encourage, cajole or reprimand, 
so COACH is an advisory system that does not interfere with the user's actions but comments 
opportunistically to help the user along." Like COACH, this PIM will serve as an advisory- 
style agent that builds a user relationship and has explicit goal of educating the individual 
(Seiker, p. 93). Similarly, the PIM will draw from an adaptive user model that selects 
appropriate advice. 

4.5 Demonstrate expert performance 

At any time in the session, the student may request the PIM to demonstrate the reasoning 
involved in the solution of a similar problem. This may serve as feedback for an incorrect 
answer and/or to "coach" the student as to why a particular strategy should be selected. 

4.6 Provide flexible feedback 

The feedback will be supplied in different chunks: from highly detailed to more global. One of 
the primary goals for the feedback is to facilitate student metacognition and reflection, and 
feedback addresses this goal in three ways. First, the PIM will be able to show the student a 
"tree" of his/her problem-solving process and show errors in performance trends. This is 
conducive to developing student reflection on his/her performance (Collins & Brown, 1987). 
Second, at appropriate times, the PIM will ask the student to explain why s/he chose a 
particular response, to get the student to analyze his/her problem-solving behavior. Third, the 
PIM will provide appropriate follow-up practice. 

One final consideration in terms of feedback is that the PIM will not provide too many insights 
and thereby annoy the student. To address this issue, part of the pedagogical task includes the 
monitoring of the timing and implementation of the advisements. The principle of minimal 
help will be the default. Also, the student will be able to select a feedback option depending on 
the amount of structure s/he desires when problem-solving. If s/he desires maximum 
interaction and feedback, the agents will gradually reduce assistance over time. 

4.7 Visually assist with problem solving 

Feedback will also facilitate learning by providing multiple representations of the information. 



It is hypothesized that learning is improved due to the formulating of multiple representations 
by the person (e.g. Lehrer, Erickson, & Connell, 1994). As a way of managing detail, Adelson 
& Soloway (1988) describe how expert software designers make mental "notes" to themselves 
about things to remember in the design process. Goel & Pirolli (1992) claim that interim 
design ideas or solutions are generated, retained, and incrementally developed — they are 
rarely forgotten or discarded. Since it is believed that these mental "notes" serve as a mental 
representation and contribute to the learning process, the student will be provided with a 
computer-based notepad to maintain his/her ideas, strategies, and mental notes. 

Additionally, as mentioned previously, the PIM will be able to show the student a "tree" of 
his/her problem-solving process to visually show errors in performance trends. 

4.8 Monitor student generation of a logic problem 

A special feature of our system is the opportunity for the student to create his/her own 
personal logical word puzzle. In this component of the system, the PIM guides the student in 
the creating and testing process. This serves as a generative learning activity that is beneficial 
because it forces the student to consciously relate the logic problem-solving strategies to 
information he/she knows is true, thereby incorporating personal knowledge while also 
problem-creating. 

4.9 Management of student control 

The setting for the logic problem-solving is constructivist because it allows the student to 
problem-solve in his/her preferred manner, and to discover truths for his/herself. In this 
constructivist environment, the agent serves as a "guide." For example, when the student 
seems to be aimlessly trying different strategies (i.e., "thrashing"), the agent will intervene. In 
one way there is program control: the program will select the next problem for the student 
based on what s/he "needs" most. This feature follows more of a traditional ITS approach. On 
the opposite end of the continuum is the component of the program which allows the student 
to create his/her own logic problem. This feature follows more of an interactive learning 
environment approach. Overall, it is a mixed approach: the student is in an interactive learning 
environment supplemented with locally intelligent agents. 



5 Properties of the Personal Intelligent Mentor 

The environment in which PIM operates is dynamically changing. Based on the task and the 
environment, there are two main properties of the PIM: agency and intelligence. 

5.1 Agency 

Agency defines the basic characteristics of the PIM with the following descriptive facets: 

Autonomy. An important characteristic of agency is autonomy, i.e., PIM is an autonomous 
software entity. It can take initiative and operate in a dynamic environment without interacting 
with a user. This is one of the major features that distinguishes an agent from other software 
programs. The PIM also has some degree of control over its own actions. 
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Social Ability. The PIM can interact with other agents. The agent might cooperate with others 
and then collectively gather the information to improve its action-goal. 

Adaptability. The PIM has an ability to change its behavior to be able to reach its goal despite 
environmental changes. 

Reactivity. The PIM responds to changes in its environment, and an agent is reactive if it is 
able to respond in a timely manner to such changes. 

Communication Ability. Explicit communication is needed among agents to enable them to 
cooperate. An agent typically communicates with others via a high-level communication 
protocol. 

5.2 Intelligence 

Intelligence introduces a stronger notion of agency. It gives an agent the ability to learn or 
understand from experience and make decisions in new situations without interacting with a 
user. 



6 Personal Intelligent Mentor as observer 

A PIM needs models of other agents, its user and the environment to construct possible future 
scenarios that it can then use to guide its actions (to give better support to the student). For 
example, if the environment is the school, there is a library, computers, teachers and students. 
If a teacher cannot solve a problem or needs to learn something to teach the students, s/he can 
go to the library and check some books out or ask his/her colleagues. In this way s/he learns 
which books are available in the environment or which teachers have more information on 
logic. It seems somewhat complicated, but today information is distributed and changing 
dynamically. Consequently, a teacher may need to update his/her knowledge, or search for 
some information to obtain a better strategy to solve the logic problem. If only one agent is 
used, it is a more static system. 

6.1 User Modeling 

PIM can construct a model of its user by tracking and monitoring its observable actions. We 
call this passive observation. 

6.2 Environment modeling 

Acting upon its environment and receiving information via percepts provided by the 
environment, a PIM creates a model of the environment in the form of an internal map. A 
utility function is then used to rank states of the environment to which the PIM's actions will 
lead. Feedback from the environment makes the PIM more independent and can play an 
important role when the PIM is making decisions autonomously. 

6.3 Agent modeling 

Proactively an agent, via ASK-TELL or REQUEST-RESPONSE protocols, may acquire and 
share other agents' knowledge and use this information to construct its model. How should 
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PIM acquire the information it needs . to construct a model of other agents? This has, we 
believe, a simple and elegant answer: the agent should presume that unknown agents are like 
itself, and it should choose to represent them as it does itself. As it learns more about them it 
only has to encode any differences that it discovers. This can make the resultant representation 
concise and efficient. 



7 Conclusion 

The primary goal of this research is to give high school students practice in thinking logically 
and developing problem-solving strategies through the process of solving logic word puzzles 
using an intelligent agent (specifically, a Personal Intelligent Mentor (PIM)) as a cognitive 
tool. We have described the basic requirements of a PIM. Our future work will include 
implementation of PIM and investigation of a PIM society. 
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Appendix A 

Sample of an easy logic word puzzle: 

Problem 

Mark loves animals. He has four pets: a newt, a Siamese cat, a Golden Retriever puppy, and a 
Piranha (fish). Mark refers to his pets as Johnny, Rusty, Scraggly, and Prickly. 

Clues: 

1) Johnny is EITHER the newt OR the Siamese cat. 

2) Rusty is NOT the cat and is NOT the newt. 

3) IF Rusty is the puppy THEN Prickly is the cat, ELSE Prickly is the newt. 

4) Both Prickly and Scraggly are NOT the piranha . 

5) Prickly is NOT the Golden Retriever. 

6) Scraggly is NOT the Siamese cat. 

What is the name of each of Mark's pet animals? 
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